Search for: All records

Demand for Machine Learning (ML) courses remains high, and educators face open questions about which prerequisites are important for student success in upper-year ML courses. Prior work has shown that instructors and students in ML courses believe that the math prerequisites and their relative recency are barriers to success, but this relationship has not been demonstrated quantitatively. In this paper, we study the link between prerequisite grades and performance in an upper-year ML course at two sites. We use linear models to study the extent to which student grades in prerequisite courses in calculus, linear algebra, statistics, and software design are predictive of student performance in the ML course. We consider the effect of additional factors like gender, first-in-family status, prior experience, comfort with mathematics, and comfort with academic English. Like prior work in many domains, and consistent with ML instructor and student perspectives, we find that prerequisite grades are predictive of ML performance. However, different combinations of prerequisites are important at different sites. Also, we find that cumulative grade point average (cGPA) in past technical and non-technical courses are as predictive of ML grade, if not more. Moreover, recency in prerequisite courses is not predictive of ML course grades in our setting. These findings suggest that general academic preparation may be as robust a predictor of ML course performance as specific math prerequisites, challenging assumptions about the role of mathematical recency and preparedness—at least as measured by grades. 

Computer science has historically presented barriers for non-native English speaking (NNES) students, often due to language and terminology challenges. With the rise of large language models (LLMs), there is potential to leverage this technology to support NNES students more effectively. Recent implementations of LLMs as tutors in classrooms have shown promising results. In this study, we deployed an LLM tutor in an accelerated introductory computing course to evaluate its effectiveness specifically for NNES students. Key insights for LLM tutor use are as follows: NNES students signed up for the LLM tutor at a similar rate to native English speakers (NES); NNES students used the system at a lower rate than NES students---to a small effect; NNES students asked significantly more questions in languages other than English compared to NES students, with many of the questions being multilingual by incorporating English programming keywords. Results for views of the LLM tutor are as follows: both NNES and NES students appreciated the LLM tutor for its accessibility, conversational style, and the guardrails put in place to guide users to answers rather than directly providing solutions; NNES students highlighted its approachability as they did not need to communicate in perfect English; NNES students rated help-seeking preferences of online resources higher than NES students; Many NNES students were unfamiliar with computing terminology in their native languages. These results suggest that LLM tutors can be a valuable resource for NNES students in computing, providing tailored support that enhances their learning experience and overcomes language barriers. 

Objectives The traditional, instructor-led form of live coding has been extensively studied, with findings showing that this form of live coding imparts similar learning to static-code examples. However, a concern with Traditional Live Coding is that it can turn into a passive learning activity for students as they simply observe the instructor program. Therefore, this study compares Active Live Coding—a form of live coding that leverages in-class coding activities and peer discussion—to Traditional Live Coding on three outcomes: 1) students’ adherence to effective programming processes, 2) students’ performance on exams and in-lecture questions, and 3) students’ lecture experience. Participants Roughly 530 students were enrolled in an advanced, CS1 course taught in Java at a large, public university in North America. The students were primarily first- and second-year undergraduate students with some prior programming experience. The student population was spread across two lecture sections—348 students in the Active Live Coding (ALC) lecture and 185 students in the Traditional Live Coding (TLC) lecture. Study Methods We used a mixed-methods approach to answer our‘ research questions. To compare students’ programming processes, we applied process-oriented metrics related to incremental development and error frequencies. To measure students’ learning outcomes, we compared students’ performance on major course components and used pre- and post-lecture questionnaires to compare students’ learning gain during lectures. Finally, to understand students’ lecture experience, we used a classroom observation protocol to measure and compare students’ behavioral engagement during the two lectures. We also inductively coded open-ended survey questions to understand students’ perceptions of live coding. Findings We did not find a statistically significant effect of ALC on students’ programming processes or learning outcomes. It seems that both ALC and TLC impart similar programming processes and result in similar student learning. However, our findings related to students’ lecture experience shows a persistent engagement effect of ALC, where students’ behavioral engagement peaks andremains elevatedafter the in-class coding activity and peer discussion. Finally, we discuss the unique affordances and drawbacks of the lecture technique as well as students’ perceptions of ALC. Conclusions Despite being motivated by well-established learning theories, Active Live Coding did not result in improved student learning or programming processes. This study is preceded by several prior works that showed that Traditional Live Coding imparts similar student learning and programming skills as static-code examples. Though potential reasons for the lack of observed learning benefits are discussed in this work, multiple future analyses to further investigate Active Live Coding may help the community understand the impacts (or lack thereof) of the instructional technique. 

Introduction: The emergence and widespread adoption of generative AI (GenAI) chatbots such as ChatGPT, and programming assistants such as GitHub Copilot, have radically redefined the landscape of programming education. This calls for replication of studies and reexamination of findings from pre-GenAI CS contexts to understand the impact on students. Objectives: Achievement Goals are well studied in computing education and can be predictive of student interest and exam performance. The objective in this study is to compare findings from prior achievement goal studies in CS1 courses with new CS1 courses that emphasize the use of human-GenAI collaborative coding. Methods: In a CS1 course that integrates GenAI, we use linear regression to explore the relationship between achievement goals and prior experience on student interest, exam performance, and perceptions of GenAI. Results: As with prior findings in traditional CS1 classes, Mastery goals are correlated with interest in computing. Contradicting prior CS1 findings, normative goals are correlated with exam scores. Normative and mastery goals correlate with students’ perceptions of learning with GenAI. Mastery goals weakly correlate with reading and testing code output from GenAI. 

Generative AI (GenAI) is advancing rapidly, and the literature in computing education is expanding almost as quickly. Initial responses to GenAI tools were mixed between panic and utopian optimism. Many were fast to point out the opportunities and challenges of GenAI. Researchers reported that these new tools are capable of solving most introductory programming tasks and are causing disruptions throughout the curriculum. These tools can write and explain code, enhance error messages, create resources for instructors, and even provide feedback and help for students like a traditional teaching assistant. In 2024, new research started to emerge on the effects of GenAI usage in the computing classroom. These new data involve the use of GenAI to support classroom instruction at scale and to teach students how to code with GenAI. In support of the former, a new class of tools is emerging that can provide personalized feedback to students on their programming assignments or teach both programming and prompting skills at the same time. With the literature expanding so rapidly, this report aims to summarize and explain what is happening on the ground in computing classrooms. We provide a systematic literature review; a survey of educators and industry professionals; and interviews with educators using GenAI in their courses, educators studying GenAI, and researchers who create GenAI tools to support computing education. The triangulation of these methods and data sources expands the understanding of GenAI usage and perceptions at this critical moment for our community.